Hierarchical Non-linear Factor Analysis and Topographic Maps

We rst describe a hierarchical, generative model that can be viewed as a non-linear generalisation of factor analysis and can be implemented in a neural network. The model performs perceptual inference in a probabilistically consistent manner by using top-down, bottom-up and lateral connections. These connections can be learned using simple rules that require only locally available information. We then show how to incorporate lateral connections into the generative model. The model extracts a sparse, distributed, hierarchical representation of depth from simpli ed random-dot stereograms and the localised disparity detectors in the rst hidden layer form a topographic map. When presented with image patches from natural scenes, the model develops topographically organised local feature detectors.